1. Field of the Invention
The present invention relates to a sample observation device, and more specifically to a sample observation device capable of acquiring a sample image of super resolution exceeding the resolution limit of an optical system.
2. Description of the Related Art
The technology of obtaining a sample image of the resolution exceeding the resolution limit of an optical image forming system (hereafter referred to as super resolution) has recently been developed and commercialized. The microscopy called “structured illumination microscopy (SIM)” is well known as one of the super resolution techniques. The SIM has been disclosed by, for example, International Publication Pamphlet No. WO 2007/043382.
In a general wide-field observation, the illuminating light is emitted to a sample and the sample is irradiated with the illuminating light as uniformly as possible. However, in the SIM, the illuminating light is modulated, and a sample is irradiated with stripes of illuminating light. Thus, the frequency of observing light used in forming an image can be shifted. Using this phenomenon, a sample image of super resolution exceeding the resolution limit of an optical image forming system can be generated.
Described below is the method of generating a sample image of super resolution.
FIGS. 1A and 1B are explanatory views about the image-forming characteristics of an optical image forming system. As illustrated in FIG. 1A, when an image is formed by an optical image forming system used in a sample observation device such as a microscope etc., the emission from one point of a sample is not projected to one point of an image forming surface, but is projected with the distribution indicated by the point spread function as a function specific to the optical image forming system. That is, as exemplified in FIG. 1B, the emission from each point of a sample is projected to the image forming surface with the distribution indicated by each point spread function.
Therefore, the optical intensity distribution IWF (x, y) of the sample image obtained by the wide-field observation, the wide-filed observation is an observation by the illumination of a fluorescent sample by uniform excitation light, is given by the convolution of PSF (x, y) and Obj (x, y) as indicated by the following equation (1) where Obj (x, y) is the distribution function of the fluorescent dye of the sample, and PSF (x, y) is the point spread function of the optical image forming system. The spatial coordinate system of the point spread function PSF and the spatial coordinate system of the distribution function Obj of the fluorescent dye are standardized using the magnification M of the optical image forming system.IWF(x,y)=Obj(x,y)PSF(x,y)  (1)
When the Fourier transform is performed on the equation (1) into an equation indicating the frequency characteristic, the equation (2) is derived. The tilde (˜) indicates a function obtained by performing the Fourier transform. That is, the equation (1) indicates the spatial intensity distribution of a sample image, and the equation (2) indicates the frequency characteristic of the sample image.ĨWF(fx, fy)=Õbj(fx, fy)×{tilde over (P)}SF(fx,fy)  (2)
FIG. 2A exemplifies the frequency characteristic of each of the distribution function of the fluorescent dye and the point spread function of the optical image forming system. FIG. 2B exemplifies the frequency characteristic of the sample image obtained by the fluorescent sample and the optical image forming system having the characteristic illustrated in FIG. 2A.
As illustrated in FIG. 2A, the frequency characteristic of the distribution function Obj of the fluorescent dye indicates the distribution in a wide frequency band. On the other hand, the frequency characteristic of the point spread function PSF has the intensity only within the range of the frequency ±fc. It indicates the transfer of only the frequency component within the range of the frequency ±fc to the image forming surface by the optical image forming system. Therefore, the frequency characteristic of the point spread function PSF is also referred to as a transfer function of an optical image forming system. Hereafter, the frequency fc is described as the cutoff frequency of the optical image forming system.
Accordingly, the frequency characteristic of the optical intensity distribution IWF of the sample image calculated by the equation (2) is limited to the frequency range of the cutoff frequency ±fc as exemplified in FIG. 2B. Thus, in the conventional wide-field observation in which a sample surface is uniformly illuminated, the resolution of a sample image is limited by the cutoff frequency of an optical image forming system, and a sample image having the resolution exceeding the resolution limit of an optical image forming system cannot be generated.
On the other hand, in the SIM, for example, a sample surface is illuminated in the sine wave pattern indicated by the following equation (3) where Illi(x) indicates the optical intensity distribution of the illuminating light on the sample surface, f0 indicates the frequency of the illumination pattern, and φi indicates the phase of the illumination pattern at the origin of x=0. Hereafter, a one-dimensional model is described for simple explanation.
                                          Ill            i                    ⁡                      (            x            )                          =                              1            +                          cos              ⁡                              [                                                      2                    ⁢                    π                    ⁢                                                                                  ⁢                                          f                      0                                        ⁢                    x                                    +                                      ϕ                    i                                                  ]                                              2                                    (        3        )            
When the Fourier transform is performed on the equation (3) to obtain the equation indicating the frequency characteristic, the equation (4) below is derived.
                                          I            ~                    ⁢                                    ll              i                        ⁡                          (              f              )                                      =                  {                                                    δ                ⁡                                  (                  f                  )                                            2                        +                                                            exp                  ⁡                                      [                                          j                      ⁢                                                                                          ⁢                                              ϕ                        i                                                              ]                                                  4                            ·                              δ                ⁡                                  (                                      f                    -                                          f                      0                                                        )                                                      +                                                            exp                  ⁡                                      [                                                                  -                        j                                            ⁢                                                                                          ⁢                                              ϕ                        i                                                              ]                                                  4                            ·                              δ                ⁡                                  (                                      f                    +                                          f                      0                                                        )                                                              }                                    (        4        )            
The optical intensity distribution Ii (x) of a sample image on a image forming surface, which is generalized in view of an optical intensity distribution of an illumination light is expressed by the equation (5), and when the Fourier transform is performed on the equation (5) for transform into the equation indicating the frequency characteristic, the equation (6) below is derived.Ii(x)={Illi(x)×Obj(x)}PSF(x)  (5)Ĩi(f)={Ĩlli(f)Õbj(f)}×{tilde over (P)}SF(f)  (6)
Therefore, the optical intensity distribution Ii on the image forming surface obtained in the SIM is expressed by the equation (7) where P0, P+, and P− respectively indicate the 0 order diffracted light component, the +1 order diffracted light component, and the −1 order diffracted light component of the optical intensity distribution on the image forming surface.
                                                                                                              I                    ~                                    i                                ⁡                                  (                  f                  )                                            =                            ⁢                                                {                                                                                                              O                          ~                                                ⁢                                                  bj                          ⁡                                                      (                            f                            )                                                                                              2                                        +                                                                                                                        exp                            ⁡                                                          [                                                              j                                ⁢                                                                                                                                  ⁢                                                                  ϕ                                  i                                                                                            ]                                                                                4                                                ·                                                  O                          ~                                                                    ⁢                                              bj                        ⁡                                                  (                                                      f                            -                                                          f                              0                                                                                )                                                                                      +                                                                                                                        exp                            ⁡                                                          [                                                                                                -                                  j                                                                ⁢                                                                                                                                  ⁢                                                                  ϕ                                  i                                                                                            ]                                                                                4                                                ·                                                  O                          ~                                                                    ⁢                                              bj                        ⁡                                                  (                                                      f                            +                                                          f                              0                                                                                )                                                                                                      }                                ·                                                                                                      ⁢                                                P                  ~                                ⁢                                  SF                  ⁡                                      (                    f                    )                                                                                                                          =                            ⁢                                                                                          O                      ~                                        ⁢                                          bj                      ⁡                                              (                        f                        )                                                              ⁢                                          P                      ~                                        ⁢                                          SF                      ⁡                                              (                        f                        )                                                                              2                                +                                                                                                    exp                        ⁡                                                  [                                                      j                            ⁢                                                                                                                  ⁢                                                          ϕ                              i                                                                                ]                                                                    4                                        ·                                          O                      ~                                                        ⁢                                      bj                    ⁡                                          (                                              f                        -                                                  f                          0                                                                    )                                                        ⁢                                      P                    ~                                    ⁢                                      SF                    ⁡                                          (                      f                      )                                                                      +                                                                            exp                      ⁡                                              [                                                                              -                            j                                                    ⁢                                                                                                          ⁢                                                      ϕ                            i                                                                          ]                                                              4                                    ·                                                                                                                      ⁢                                                O                  ~                                ⁢                                  bj                  ⁡                                      (                                          f                      +                                              f                        0                                                              )                                                  ⁢                                  P                  ~                                ⁢                                  SF                  ⁡                                      (                    f                    )                                                                                                                          ≡                            ⁢                                                                    1                    2                                    ⁢                                                                                    P                        ~                                            0                                        ⁡                                          (                      f                      )                                                                      +                                                                            exp                      ⁡                                              [                                                  j                          ⁢                                                                                                          ⁢                                                      ϕ                            i                                                                          ]                                                              4                                    ⁢                                                                                    P                        ~                                            +                                        ⁡                                          (                      f                      )                                                                      +                                                                            exp                      ⁡                                              [                                                                              -                            j                                                    ⁢                                                                                                          ⁢                                                      ϕ                            i                                                                          ]                                                              4                                    ⁢                                                                                    P                        ~                                            -                                        ⁡                                          (                      f                      )                                                                                                                              (        7        )            
As illustrated by the equation (7), the optical intensity distribution Ii of the sample image on the image forming surface includes the ±1 order diffracted light component in addition to the 0 order diffracted light component by illuminating the sample surface by the sine wave pattern.
FIG. 3A exemplifies the frequency characteristic of each order diffracted light component on the sample surface generated by the sine wave illumination pattern and the point spread function of the optical image forming system. FIG. 3B exemplifies the frequency characteristic of each order diffracted light component on the image forming surface.
As exemplified in FIG. 3A, the frequency characteristic of the +1 order diffracted light component and the frequency characteristic of the −1 order diffracted light component on the sample surface are distributed respectively with the shift of the frequencies f0 and −f0 with respect to the frequency characteristic of the 0 order diffracted light component on the sample surface. However, the frequency characteristic of the optical intensity distribution on the image forming surface is a sum of the products of the frequency characteristic of each order diffracted light component and the frequency characteristic of the PSF of the optical image forming system on the sample surface. Therefore, the frequency characteristic of the optical intensity distribution of the sample image is limited to the frequency band within the range of the cutoff frequency ±fc of the optical image forming system as in the case of uniform illumination as exemplified in FIG. 3B.
Then, in the SIM, each order diffracted light component on the image forming surface exemplified in FIG. 3B is individually calculated, and a super resolution image including the frequency exceeding the cutoff frequency fc is generated using the calculation results. Hereafter, the super resolution image obtained by the SIM is specifically described as a SIM reconstructed image.
Practically, using a plurality of sample images captured in different initial phases φi, each order diffracted light component on the image forming surface is calculated by solving the simultaneous linear equations expressed by the each order diffracted light components P0, P+, and P− on the image forming surface as variables.
Next, the shifts of the frequencies with respect to the 0 order diffracted light component (hereafter referred to as an origin shift) is corrected for the ±1 order diffracted light component P+ and P−0 among the calculated diffracted light components P0, P+, and P− on the image forming surface. Practically, as indicated by the equations (8) and (9), the frequencies are shifted by the frequencies −f0 and f0. FIG. 4 exemplifies each order diffracted light component after the origin shift. P′+ and P′− indicate the ±1 order diffracted light component of the optical intensity distribution on the image forming surface after the origin shift.{tilde over (P)}′+(f)≡{tilde over (P)}+(f+f0)=Õbj(f)·{tilde over (P)}SF(f+f0)  (8){tilde over (P)}′−(f)≡{tilde over (P)}−(f−f0)=Õbj(f)·{tilde over (P)}SF(f−f0)  (9)
Furthermore, the calculated 0 order diffracted light component is added to the product of the ±1 order diffracted light component after the origin shift and the weight w, thereby generating a SIM reconstructed image by the image processing.
The equation (10) indicates the frequency characteristic of the SIM reconstructed image. In the equation, ISIM indicates the intensity distribution of the SIM reconstructed image, and PSFSIM indicates the point spread function of the SIM reconstructed image.
                                                                                                              I                    ~                                    SIM                                ⁡                                  (                  f                  )                                            ≡                            ⁢                                                                                          P                      ~                                        0                                    ⁡                                      (                    f                    )                                                  +                                  w                  ⁢                                      {                                                                                                                        P                            ~                                                    +                          ′                                                ⁡                                                  (                          f                          )                                                                    +                                                                                                    P                            ~                                                    -                          ′                                                ⁡                                                  (                          f                          )                                                                                      }                                                                                                                          =                            ⁢                                                                                          P                      ~                                        0                                    ⁡                                      (                    f                    )                                                  +                                  w                  ⁢                                      {                                                                                                                        P                            ~                                                    +                                                ⁡                                                  (                                                      f                            +                                                          f                              0                                                                                )                                                                    +                                                                                                    P                            ~                                                    -                                                ⁡                                                  (                                                      f                            -                                                          f                              0                                                                                )                                                                                      }                                                                                                                          =                            ⁢                                                                    [                                                                                            P                          ~                                                ⁢                                                  SF                          ⁡                                                      (                            f                            )                                                                                              +                                              w                        ⁢                                                  {                                                                                                                    P                                ~                                                            ⁢                                                              SF                                ⁡                                                                  (                                                                      f                                    +                                                                          f                                      0                                                                                                        )                                                                                                                      +                                                                                          P                                ~                                                            ⁢                                                              SF                                ⁡                                                                  (                                                                      f                                    -                                                                          f                                      0                                                                                                        )                                                                                                                                              }                                                                                      ]                                    ·                                      O                    ~                                                  ⁢                                  bj                  ⁡                                      (                    f                    )                                                                                                                          ≡                            ⁢                                                P                  ~                                ⁢                                                                            SF                      SIM                                        ⁡                                          (                      f                      )                                                        ·                                      O                    ~                                                  ⁢                                  bj                  ⁡                                      (                    f                    )                                                                                                          (        10        )            
As illustrated in FIG. 5, the frequency characteristic of the point spread function PSFSIM of the SIM reconstructed image has the intensity within the range of the frequency band ±(fc+f0) obtained by adding the frequency f0 of the illumination pattern to the cutoff frequency fc of the optical image forming system. That is, the cutoff frequency of the SIM reconstructed image is fc+f0, and the frequency band exceeding the cutoff frequency fc of the optical image forming system can be transferred.
Thus, according to the SIM, the frequency band exceeding the cutoff frequency of the optical image forming system can be transferred, and a sample image of super resolution exceeding the resolution limit of the optical image forming system can be generated. In addition, by changing the weight w, the visibility of the super resolution component exceeding the cutoff frequency of the optical image forming system can be adjusted.
Furthermore, International Publication Pamphlet No. WO 2007/043314 has disclosed the configuration arranging the spatial modulation element for generating structured illumination at an intermediate image position, and demodulating and capturing the intermediate image of a sample through the spatial modulation element.